1. Field of the Invention
This invention relates generally to image processing methods and apparatus. More particularly, this invention relates to image processing methods and apparatus for interpolation of image data for adapting to various edges and shapes that might interfere with the uniform illumination and cause color shifts. Even more particularly, this invention relates to image processing methods and apparatus that employ a second order derivative calculated at shift invariant points of a mosaic structured color element array to determine color values independent of the location chosen at intervals of maximum spatial sampling frequency, where the mosaic structured color element array does not have a fully populated shift invariant matrix.
2. Description of Related Art
A digital image is an electronic signal representing the intensity or intensity of light reflected or originating from an object impinging upon a sensor. The light is converted within the sensor to an electronic signal. In an image sensor array the electronic signal is contains information detailing the intensity of the light impinging upon a two-dimensional array. Thus the electronic signal contains the intensity of each point having a sensor within the array as defined as a function of two spatial variables. Further, each point having a sensor is considered a sampling point and the space between each of the sensors determines the maximum spatial sampling frequency of the image. Thus projected images of these sensor outputs such as photographs, still video images, radar images, etc. are a function of the spatial variables (x, y), therefore the image intensity is defined as f(x,y).
U.S. Pat. No. 6,822,758 (Morino) describes an image processing method for improving a defective image (degraded image) using color interpolation and optical correction.
“Pixel-Level Image Fusion: The Case of Image Sequences”, Rockinger, et al, Proceedings of SPIE (The International Society for Optical Engineering), Signal Processing, Sensor Fusion, and Target Recognition VII, Vol. 3374, pp.: 378-388, July 1998, provides a pixel-level image sequence fusion with an approach based on a shift invariant extension of the 2D discrete wavelet transform. The discrete wavelet transform yields an over-complete and thus shift invariant multi-resolution signal representation.
“Method of Color Interpolation in A Single Sensor Color Camera Using Green Channel Separation”, Weerasinghe, et al, IEEE International Conference on Acoustics, Speech, and Signal Processing, 2002, Vol.: 4, pp.: IV-3233-IV-3236 presents a color interpolation algorithm for a single sensor color camera. The proposed algorithm is especially designed to solve the problem of pixel crosstalk among the pixels of different color channels. Inter-channel crosstalk gives rise to blocking effects on the interpolated green plane, and also spreading of false colors into detailed structures. The proposed algorithm separates the green channel into two planes, one highly correlated with the red channel and the other with the blue channel. These separate planes are used for red and blue channel interpolation.
“The Canonical Correlations of Color Images and Their Use for Demosaicing”, Hel-Or, Hewlett Packard Laboratories, HPL-2003-164R1, Feb. 23, 2004, found: Mar. 29, 2006 at www.hpl.hp.com/techreports/2003/HPL-2003-164R1.pdf, describes a demosaicing technique that is derived directly from statistical inferences on color images for demosaicing color image de-noising, compression, and segmentation. The technique presents a new Bayesian approach that better exploits the spectral dependencies in color images. It takes advantage of the fact that spatial discontinuities in different color bands are correlated and that this characteristic is efficiently exposed using the Canonical Correlation Analysis (CCA). The CCA scheme aims attending the optimal representation of each color band such that color plane correlation is maximized.
“Local Image Reconstruction and Sub-pixel Restoration Algorithms”, Boult et al, Computer Vision, Graphics, and Image Processing: Graphical Models and Image Processing, Vol.: 55, No.: 1, 1993, pp.: 63-77, Academic Press, Inc., Orlando, Fla., introduces a new class of reconstruction algorithms that treat image values as area samples generated by non-overlapping integrators. This is consistent with the image formation process, particularly for CCD and CID cameras. Image reconstruction is a two-stage process: image restoration followed by application of the point spread function (PSF) of the imaging sensor.
“Image Capture: Modeling and Calibration of Sensor Responses and Their Synthesis from Multispectral Images”, Vora, et al, Hewlett Packard Laboratories, HPL-98-187, found Mar. 29, 2006 at http://www.hpl.hp.com/techreports/98/HPL-98-187.pdf, models for digital cameras, methods for the calibration of the spectral response of a camera and the performance of an image capture simulator. The general model underlying the simulator assumes that the image capture device contains multiple classes of sensors with different spectral sensitivities and that each sensor responds in a known way to light intensity over most of its operating range.